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PREFACE 

This  paper  Is  concerned  vlth  a  portion  of  RAND's  stvidy  of  automatic 
checkout  equipment,  which  was  Initiated  late  In  19^8  at  the  request  of 
the  Air  Force.  The  project  objective,  as  stated  by  Gen.  S.  E.  Anderson, 
then  commander  of  ARDC,  was  .  .to  establish  the  basic  philosophy, 
concepts,  and  peurameters  for  application  of  automatic  test  and  checkout 
equipment  to  Improve  the  operational  readiness  and  eiqployment  of  weapon 

systems. ".,^The  primary  activity  then  was  to  develop  concepts,  design 

1 ... 

methods  and  techniques,  and  decision  aids  to  help  make  these  test  equip¬ 
ments  and  systems  more  effective  —  In  short,  to  help  make  the  best  use 
of  test  eqvilpment  dollars  to  get  the  best  weapon  system  performance. 

RAND  Research  Memorandum  RM-2750,  "An  Omnibus  of  Brief li:g  Papers  on 
Analysis  of  Automatic  Checkout  Equipment  and  Aids  to  Its  Design,"  contains 
a  conprehenslve  description  of  the  project  results  and  a  bibliography  of 
the  ten  detedled  project  reports. 
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INTRODUCTION 

This  discussion  concerns  some  economic  aspects  of  the  use  of  test 
equipment,  particularly  automatic  checkout  equipment.  In  a  system  context. 
The  object  of  the  study  was  to  develop  a  method  for  determining  preferred 
Inspection  Intervals,  considering  both  readiness  and  costs,  at  a  system 
level. 

The  question  of  how  often  a  system  should  be  tested  comes  up  early  In 
the  consideration  of  checkout  equipment.  It  has  a  direct  bearing  on  how 
much  testing  must  be  done,  and  utilization  Is  one  of  the  Important  factors 
in  the  decision  to  use  manual,  semi-automatic,  or  fully  automatic  test 
equipment.  It  Is  also  important  in  determining  how  much  equipment  Is 
required,  and  Its  relation  to  the  rest  of  the  system. 

In  explaining  the  method.  I'll  first  show  how  readiness  and  costs  are 
affected  by  the  frequency  of  checkouts.  Then  I'll  show  one  way  to  achieve 
a  desirable  balance  between  them.  I'll  be  talking  specifically  about 
systems  whose  failures  occur  exponentiedly,  or  In  a  purely  random  fashion. 


*Any  views  expressed  in  this  paper  are  those  of  the  author.  They 
should  not  be  interpreted  as  reflecting  the  views  of  The  RAND  Corporation 

or  the  official  opinion  or  policy  of  any  of  its  goveriimentel-  or  private 
research  sponsors.  Papers  are  reproduced  by  The  RAND  Corporation  as  a 
courtesy  to  members  of  its  staff. 
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It  has  been  shown  that  many  vacuum  tubes,  electronic  piece-parts,  airborne 
radars,  fire-control  systems,  and  alr-to-edr  missiles  have  this  very  common 
fedl\ire  characteristic,  sometimes  referred  to  as  a  constant  failure  rate. 

READINESS 

Let's  consider  a  system  that  is  to  be  maintained  in  a  condition  of 
static  alert  for  a  long  time.  If  the  system  as  a  whole  is  subject  to  random 
failures,  and  we  start  with  a  good  system,  its  survival  probability  will 
be  an  exponential  curve,  decreasing  with  time  (Fig.  l).  T5ie  particular 
e;q)onentlal  c\irve  shown  is  for  a  system  with  a  mean  time  to  failure  of 
285  days,  or  about  9*1/2  months,  and  represents  some  data  obtained  on  em 
early  alr-to-air  missile. 

If  we  have  a  force  of,  say,  100  of  these  systems,  and  they  aie  ballistic 
missiles,  the  same  curve  can  be  used  to  show  how  many  of  these  100  systems 
are  expected  to  be  good  at  any  time.  At  the  end  of  10  days,  3  or  4  will 
have  fedled,  although  we  woiildn't  know  Just  which  ones.  By  100  days,  about 
30  would  have  failed,  and  so  on.  Now  if  we  inspect  these  100  systems  each 
100  days  and  repair  the  bad  ones  (instantaneously,  for  the  moment)  the 
expected  number  of  good  systems  would  follow  the  lower  curve  of  Fig.  2. 

If  instead  of  checking  all  100  at  once,  we  were  to  stagger  the  inspections 
so  that  we  checked  a  different  one  each  day  until  all  100  had  been  covered, 
and  then  started  ove"  again,  the  expected  number  of  good  systems  would  then 
be  represented  by  the  average  height  of  the  curve,  or  about  84  systems, 
conpared  to  the  extremes  of  70  (just  before  checkout)  and  100  (just  after 
checkout)  for  the  case  where  we  check  all  of  them  at  the  same  time.  If  we 
Inspect  and  repair  each  10  days  Instead  of  100,  the  situation  Is  represented 
by  the  upper  curve,  with  an  average  of  better  than  98  systems  ready,  between 
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extremes  of  approximately  96  and  ICX).  Inspections  every  100  days  give 
84  per  cent  readiness,  while  inspections  every  10  days  give  98  cent. 

In  Fig.  3>  the  top  curve  shows  more  clearly  the  relationship  between 
the  average  number  of  ready  missiles  and  the  inspection  interval  (large 
dots  indicate  the  two  previously  discussed  cases).  This  curve  indicates 
that  we  should  like  to  inspect  as  frequently  as  possible.  However,  the 
equation  represented  by  this  curve  was  not  the  most  useful  or  realistic 
one;  it  ignored  some  potentially  important  factors,  as  will  be  seen. 

SOME  IMPORTANT  FACTORS 

Bie  first  factor  is  the  time  required  to  make  a  repair.  Hie  second 
c\u:ve  of  Fig.  3  shows  the  effect  of  a  repair  time  of  one  day.  In  cases 
where  the  repair  time  is  small  by  conq>arison  with  the  mean  time  to  f allium, 
the  effect  is  not  very  great. 

Now  let's  consider  that  the  insi>ection  activity  itself  can  actually 
cause  failiu’es,  in  addition  to  discovering  the  ones  that  occurred  between 
inspections  (the  so-called  "tinkeritls”  effect).  The  third  curve  of  Fig.  3 
shows  how  an  inspection  stress  that  causes  0.1  probability  of  failire  affects 
the  situatlcn.  This  probability  means  that  one  time  in  ten,  a  checkout  will 
cause  a  failure  in  an  otherwise  good  system.  Now  for  the  first  time  we  find 
that  there's  such  a  thing  as  checking  too  often,  and  there's  some  preferred 
interval,  which  in  this  exanqjle  is  8  days,  if  the  object  is  to  have  the 
most  good  systems. 

Finally,  let's  also  assume  that  the  missile  is  out  of  service  or  off 
alert  for  1  hour  while  an  inspection  is  being  made.  The  last  curve  of 
Fig.  3  shows  some  fxirther  decrease  in  readiness,  and  a  slight  upward  shift 
in  the  preferred  interval,  to  9  days.  Note  that  automatic  checkout  equipment 


Fig. 3 — Ready  rate  versus  inspection  interval 


can  reduce  all  three  of  these  factors  by  neains  of  Increased  testing  speed. 

^Is  Increased  speed  reduces  testing  and  repair  times  directly,  and  nay  edso 
reduce  the  exposure  to  stresses  during  testing. 

POSTS 

Now  let's  try  to  find  out  how  the  checkout  Interval  affects  costs.  A 
slnple  but  fairly  comprehensive  procedvire  Is  to  assume  a  linear  cost  equation. 
^Is  means  that  each  checkout  costs  a  fixed  amount  of  mon^y,  regardless  of 
hov  many  are  made.  Likewise,  each  repair  costs  some  average  amount  Indepen¬ 
dent  of  quantity.  Total  costs  are  then  equal  to  the  sum  of  (l)  the  nuniber 
of  inspections  multiplied  y  the  cost  of  each  one,  (2)  the  number  of  repairs 
multiplied  by  the  cost  of  a  repair,  and  (3)  edl  other  costs,  which  are 
presvunably  independent  of  the  checkout  and  repair  process.  Incidentally, 
adding  sophistications  to  this  slnple  cost  model  doesn't  change  the  results 
very  much.  Figure  U  shows  costs,  confuted  with  the  linear  equation,  as  a 
function  of  time  between  inspections.  Costs  of  inspection  and  of  the  repair 
of  fall\jres  caused  by  inspections  are  closely  related  to  the  frequency  of 
the  inspections.  Qlie  cost  of  the  fallvires  that  occur  between  Inspections  also 
depends  on  the  inspection  interval.  This  is  because  it  is  eiss\amed  that  only 
one  failure  can  occur  between  inspections.  (If  you  don't  look  for  a  failure, 
you  don't  find  it  and  don't  fix  it.) 

READINESS  VERSUS  COSTS 

Figure  5  shows  both  readiness  and  cost  as  ftmctions  of  time  between 
inspections  for  an  example  in  which  the  parameters  are  from  a  ciirrent  Air  Force 
system.  Paradoxically,  the  best  readiness  is  achieved  at  nearly  the  highest 
cost,  and  the  lowest  cost  at  the  worst  readiness.  Moving  to  the  right  trades 
checkout  and  repedr  costs  for  extra  missiles. 
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The  trade-off  between  capability  (as  measured  here  by  readiness)  and 
costs  is  frequently  encountered  in  military  studies.  The  desirable  goal 
depends  on  when  ihe  analysis  is  made,  and  what  restraints  are  placed  on 
system  planning.  I  used  a  relatively  sin^sle  method  for  quantifying  this 
trade-off. 

THE  LONG-RANGE  PROBLEM 

The  situation  which  is  easiest  to  handle  and  which  seems  to  lend  Itself 
best  to  optimization  is  long-range  planning  for  a  weapon  system,  in  which 
we  can  select  not  only  the  checkout  Interval,  but  other  things,  such  as  the 
number  of  weapons,  and  the  capacity  of  the  checkout  and  repair  facilities. 

For  this  situation,  two  equally  valid  optimizing  objectives  can  be  proposed: 

(1)  Given  a  reqialred  alert  capability,  pick  the  number  of  weapons  and 
the  level  of  support  that  will  achieve  it  at  the  lowest  cost;  or 

(2)  Given  a  budget  limitation,  choose  the  number  purchased  and  the 
support  level  that  will  give  the  most  alert  capability. 

In  either  case,  the  long-r\m  goal  is  to  get  the  most  ready  weapons  per 
dollar  spent. 

The  third  curve  of  Fig.  5  shows  how  the  readiness-to-cost  ratio  Is 
eiffected  by  the  checkout  Interval.  The  maximum,  at  55  days,  is  achieved 
somewhere  between  the  9-day  Interval,  which  maximizes  readiness  and  the 
unrealistic  "never  check"  policy,  which  minimizes  cost. 

Three  important  points  con  be  summarized  here.  Ihe  first  is  that 
several  factors  may  be  In^jortant  in  determining  the  preferred  checkout 
Interval.  Second,  there  may  be  two  "best"  inspection  intervals:  one  is 
the  55-day  interval  during  normal  peacetime  alert;  the  other  the  9-day 
Interval  for  short  periods  of  time  during  high  tension  or  potential  emergency. 
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Bils  would  involve  lengthening  work  shifts  and  other  emergency  procedures. 

The  Increased  cost  incurred  in  moving  toward  a  9"day  Interval  might  well 
be  borne  for,  spy,  a  few  weeks  during  a  weapon's  lifetime,  and  the  tengjorary 
added  capability  can  be  as  much  as  6  per  cent  in  this  example.  Uilrd,  you 
can  see  that  if  you  fall  to  give  some  close  attention  to  the  frequency  of 
inspections,  you  may  unknowingly  wander  off  the  path  in  either  of  two 
damaging  directions:  you  can  end  up  practicing  the  false  economy  of  con¬ 
ducting  too  few  checkouts,  in  which  case  many  of  your  expensive  systems 
are  useless;  or  you  can  fall  into  the  all- too- common  hazard  of  conducting 
too  frequent  checkouts,  where  you  not  only  pay  unnecessarily  high  costs 
but  actually  get  fewer  ready  systems  than  you  might  otherwise. 

Figure  6  shows  the  kind  of  do-lt-yo\irself  nomogram  that  can  be  used 
to  summarize  this  method  for  determining  inspection  Intervals  as  functions 
of  the  readiness  characteristics  and  costs.  It  is  constructed  from  the 
results  of  several  thousand  Involved  congjutatlons,  performed  on  a  digital 
computer,  ai;d  permits  you  to  use  the  basic  method  with  Just  a  little  slnqjle 
arithmetic.  The  inputs  are  the  mean  time  to  failure,  shown  along  the  hori¬ 
zontal  axis,  and  a  composite  cost  para^^ter  (CP).  I^ie  output  here  is  the 
inspection  interval  that  gives  the  most  ready  systems  per  dollar. 

I'd  like  to  point  out  some  things  about  the  sensitivity  of  the  results 
to  the  accuracy  of  the  Inputs.  You  may  be  aware  that  cost  estimates  are 
generally  inaccurate  for  advanced  weapon  systems  --  usuedly  low.  Fortunately, 
the  reqviired  estimates  here  are  not  dollar  amounts,  but  rather  the  ratio 
of  one  cost  to  another  within  the  system,  eind  these  axe  much  less  subject 
to  estimating  errors. 

Notice  that  when  the  mean  life  changes  a  hundredfold,  from  30  days  to 
3000,  the  resulting  inspection  Interval  goes  up  only  tenfold  --  from  one 
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(s^op)  suo!i38dsui  uasMiaq  aujij. 


versus 


day  to  10,  or  from  10  days  to  100,  for  exaniple.  This  means  that  the  check¬ 
out  Interval  is  relatively  insensitive  to  errors  in  the  estimate  of  mean 
life,  and  is  even  less  sensitive  to  errors  in  any  other  Inputs,  so  much  so 
that  many  of  them  can  be  Ignored  in  a  diagram  like  this.* 

PROBLEM  AEEAS 

We  will  now  consider  some  of  the  problem  areas  in  using  this  method. 

First,  one  must  have  estimates  of  cost  and  reliability,  though  I've 
implied  that  they  don't  have  to  be  very  good  ones. 

Budget  or  requirement  changes,  if  they  occur  in  the  planning  stage, 
have  little  or  no  effect  on  the  determination  of  the  best  inspection 
intervals.  However,  if  they  occur  after  commitments  are  made  for  Inspection 
and  repair  capacities,  it  may  not  be  possible  to  achieve  the  most  desirable 
arrangement;  then  one  can  only  choose  among  the  alternatives  that  still 
exist,  and  the  analysis  used  here  can  be  helpful  for  that  purpose. 

Other  factors  external  to  the  system  can  also  restrict  the  freedom 
to  choose  inspection  and  repair  capacities,  and  thus  prevent  the  implementation 
of  the  best  solution.  For  example,  disarmament  agreements  may  limit  total 
numbers  of  weapons,  facilities,  or  manpower  in  some  presently  unknown 
manner. 

So  far,  I've  considered  only  a  pre-attack  situation.  However,  this 
same  method  can  be  used  in  an  analysis  that  accounts  for  ground  vulnerability 
and  sencond- strike  capability. 

OTHER  jVFPLICATIONS 

This  method  also  permits  us  to  make  comparisons  of  periodic  and  monitored 
checkout  on  a  cost-effectiveness  basis.*  If  we  make  some  plausible 

*For  more  detailed  treatment  see  Kamlns,  M. ,  Determining  Checkout 
Intervals  for  Systems  Subject  to  Random  Failures,  Ihe  RAND  Corporation, 

Research  Memorandum  FiM-2578,  June  15,  i960. 
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asEumptlonj  atout  the  characteristics  of  electronics  systems,  for  example, 
we  can  show  that  monitored  or  continuous  checkout  is  advantageous  when  (l) 
the  mean  time  to  failure,  (2)  the  cost  of  a  repair,  and  (3)  the  ratio  of 
the  operating  failure  rate  to  the  standing  failure  rate  are  all  relatively 
low.  'Tiey  are  most  likely  to  be  low  in  relatively  unreliable  subsystems 
(low  mean  time  to  fallui’e),  which  are  modularized  (for  ease  of  repair), 
and  use  solid-state  components  with  generous  derating  (so  that  operating 
stresses  are  not  much  higher  than  standing  stresses  ). 

The  -lethod  as  described  was  developed  for  a  complete  weapon  system, 
but  can  also  be  applied  to  portions  of  a  system. 


